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Abstract 

This study examined and quantified the maintenance of reading comprehension effects 
from a fourth grade intervention for students with reading difficulties across the summer into the 
fall of fifth grade. Fourth grade students with reading comprehension difficulties were randomly 
assigned to a multi-component reading intervention (Passport) or to typical school services. 
Students in the treatment received daily intervention in small groups of 4-7 students across the 
fourth grade school year. Students (n = 269) were followed into their fifth grade year. Groups 
were equivalent at baseline and the treatment group significantly outperformed the comparison 
group at posttest. Students in the treatment continued to grow in reading comprehension over the 
summer at a rate of approximately 25% of the original growth, suggesting partially fleeting 
effects of the intervention. Treatment students maintained the higher levels of reading 


comprehension from the end of the fourth grade intervention to the beginning of fifth grade. 
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Retention of Reading Intervention Effects from Fourth to Fifth Grade for Students 
with Reading Difficulties 

Response to intervention (RTI) models, and multi-tiered systems of support (MTSS) are 
designed to provide a structured, databased system for educators to deliver appropriate levels of 
instructional support for students through the grade levels. In these models, supplemental 
interventions are provided to students who do not make adequate progress with grade level, 
classroom instruction. A large body of research substantiates the value of supplemental, small 
group reading intervention of varying intensity levels across the elementary grades for students 
with reading difficulties (e.g., Gersten et al., 2008; O’Connor, Fulmer, Harty, & Bell, 2005; 
Vadasy & Sanders, 2009; Wanzek et al., 2017; Wanzek, Wexler, Vaughn, Cuillo, 2010). 
Gersten et al. (2008) noted the strong evidence for intensive, systematic instruction on 
foundational skills for students with reading difficulties in the early elementary grades. Research 
also supports small group, supplemental reading interventions provided in the upper elementary 
grades with positive effects for word recognition and comprehension interventions, and some of 
the highest effects noted for multicomponent interventions (Wanzek et al., 2010). The majority 
of the research on these interventions has highlighted accelerated learning for students with 
reading difficulties within a grade level as an important effect. Studies reporting long-term 
effects of reading interventions note decreased, but sustained effects on average (Suggate, 2014). 
Less studied, are the specific effects of these interventions from one grade level to the start of the 
next grade level when instruction often discontinues for the summer months. 

Summer breaks can provide difficulties for the continuous implementation of RTI/MTSS 
models, wherein student response to an intervention provides the required information for future 


instructional decisions. Entwisle, Alexander, and Olson (2000) discuss a faucet theory that is 
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useful to RTI/MTSS model implementation. In this theory, the school year offers a resource 
faucet that is turned on with all children having access to the resources needed to make learning 
gains. In RTI/MTSS models, these resources include supplemental interventions and databased 
decision making regarding the intensity level of instruction a student requires for learning. 
However, when school is not in session, such as summer break, the resource faucet is turned off, 
and students with learning difficulties no longer have access to interventions delivered in school 
to allow them to continue making gains, or perhaps even maintain their current gains. Thus, 
instructional decisions for a student may look different from the end of fourth grade to the 
beginning of the fifth grade simply because the faucet was turned off for several months; a fact 
that is not explicitly part of RTI/MTSS model decision making. In this study, we sought to 
quantify the extent to which fourth grade students with reading difficulties retained their learning 
over the summer after participating in a successful supplemental reading intervention. That is, to 
what extent did fourth grade students with reading difficulties maintain, accelerate, or decelerate 
their learning advantage from fourth grade to fifth grade when intervention resources were not 
available. 
Retention of Learning 

Retention of learning requires not only memory of the information learned, but the ability 
to retrieve the information at appropriate times (Bennett & Rebello, 2012). As researchers and 
educators we seek instructional practices that will allow students to effectively learn new 
knowledge and skills, for example to read and understand text, as well as successfully retain that 
information for application and future learning, such as reading more complex text or reading to 
learn in a content area. McCoach and Kaniskan (2010) describe three possible retention 


scenarios following successful instruction. The first is a persistent effect, wherein the instruction 
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increases student learning and that increase persists over time. For example, consider a reading 
intervention that is found to accelerate typical student learning by a half a standard deviation. 
This intervention effect is persistent when these students are still a half a standard deviation 
ahead even months after the intervention was completed. Alternatively, the accelerated learning 
may have allowed students to continue to learn at greater rates and the persistent effect exists 
because they continue to grow in their reading skills at a similar or faster rate even after the 
intervention ended, demonstrating they are more than a half a standard deviation ahead months 
after the intervention. A second possible outcome is a fully fleeting effect of intervention. 
Students increased their learning in an intervention, but those accelerated effects disappeared 
over time. In this case, students are no longer ahead months after the intervention was completed 
because the initial accelerated learning completely dissipated. A third possibility is that the 
learning effects do not fully persist over time, but do not fully dissipate either; only some degree 
of the effects of instruction were fleeting. For example, 25% of the effects are lost over time but 
75% of the effects persist. In our previous example, this would mean that, over time, students 
were no longer a half a standard deviation ahead in their learning but they were still a little more 
than a third of a standard deviation ahead in their learning, demonstrating the effect of the 
intervention has not completely disappeared but that there were some fleeting effects. McCoach 
and Kaniskan’s retention framework provides us a way to quantify the retention of effects of a 
successful intervention over the summer break between grades. 

Retention of learning from grade to grade (i.e. retention of learning over the summer break) 
for students with reading difficulties who receive supplemental reading interventions is a key, 
albeit under-researched, area of interest in RTI/MTSS model implementation. Understanding the 


amount of accelerated learning that can be retained into the next grade level after a successful 
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supplemental reading intervention can assist schools in selecting appropriate interventions to 
help students close the achievement gap over time rather than spending significant time 
regaining learning momentum. Undoubtedly, students who have reading difficulties can close 
the gap with grade level expectations efficiently if they: 1) participate in effective, research- 
based interventions that are designed to accelerate learning, 2) are able to accelerate learning to 
meet or close the gap with grade level expectations, and 3) can retain high amounts of that 
learning into the next grade level when the intervention resource faucet is turned on again. 

On average, students do lose ground in academics over the summer. Average learning loss 
over the summer is approximately one month of learning, with some students experiencing more 
significant loss (Cooper, Nye, Charlton, Lindsay, & Greathouse, 1996). Students from low- 
income backgrounds, in particular, tend to lose more ground in reading than their higher income 
peers (Alexander, Entwisle, & Olson, 2001; Benson & Borman, 2010). These effects seem to 
accumulate through the elementary grades with students from low-income backgrounds falling 
further and further behind their peers (Entwisle et al., 1997). Additionally, as grade level 
increases, so does summer learning loss (Hill, Bloom, Black, & Lipsey, 2007). However, ina 
summary of basic learning research, McCombs et al. (2011) noted that it is also the case that 
higher amounts of initial learning lead to longer retention, which is one reason researchers and 
educators seek interventions with the highest effects. We might expect that students with 
learning difficulties, who have lower levels of initial learning, may experience larger than 
average losses between grades. Yet, the retention effects of a successful reading intervention 
have not been quantified in the research for students with reading difficulties. We sought to 
examine the retention of intervention effects from one grade level for students with reading 


difficulties as they began the next grade level. 
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Passport Reading Intervention 

We recently conducted a study examining a widely used, multi-component reading 
intervention for fourth grade students with reading difficulties (Wanzek et al., 2017). Previous 
research for students with reading difficulties in the upper elementary grades suggests the 
possible importance of addressing multiple reading components for these older students 
(O’Connor et al., 2002; Ritchey, Silverman, Montanaro, Speece, & Schatschneider, 2012; 
Therrien, Wickstrom, & Jones, 2006; Vadasy & Sanders, 2008; Wanzek & Roberts, 2012; 
Wanzek et al., 2010). The Passport intervention (Voyager Sopris Learning, 2008) applies 
principles of behavioral learning theory and cognitive psychology (Flavell, 1992; Palincsar & 
Brown, 1984), providing explicit instruction and strategies for reasoning in the foundational 
components of reading (e.g., decoding, word reading) as well as reading comprehension and 
vocabulary. The first 6 weeks of the instruction emphasized the foundational components with 
application to text reading and reading for understanding, while the rest of the intervention 
provided brief instruction in foundational skills and emphasized text reading and reading 
comprehension practices. The lessons built in a systematic process and provided explicit 
instruction in new reading practices with modeling and scaffolded instruction to assist students in 
gaining mastery. 

We examined 451 students scoring at or below the 30" percentile in reading comprehension 
who were randomly assigned to the multi-component reading intervention (Passport) or to 
comparison (typical school services). Students in the treatment group received the Passport 
intervention for 30 min daily in small groups of 4-7 students across the school year. At the end 
of the school year, students in the treatment performed similarly to students in the comparison 


group in word reading and vocabulary, but significantly outperformed students in the comparison 
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group on reading comprehension (ES = 0.38; Wanzek et al., 2017). Thus, on average, students 
with reading difficulties receiving the Passport intervention ended the school year with higher 
levels of reading comprehension than students with reading difficulties receiving typical school 
services. 
Purpose 
We were interested in the retention of these accelerated reading comprehension effects 
resulting from fourth grade intervention into the following grade level, fifth grade. We sought to 
extend the existing literature in two ways: (1) examine the retention of accelerated learning from 
a school year intervention during the summer months (retention of intervention effects from the 
end of one grade to the beginning of the next when intervention resources are turned off) 
specifically for students with reading difficulties who had completed a successful intervention, 
and (2) quantify the retention of these intervention effects into the next grade level using the 
McCoach and Kaniskan’s framework. Specifically, we addressed the following research 
question: How much of the initial treatment effect on reading comprehension for fourth grade 
Passport intervention remains at the start of fifth grade? We hypothesized a partially fleeting 
effect on average for these students with reading difficulties, as they did not receive summer 
intervention between their fourth and fifth grade year. 
Method 

Participants 

Our sample comprises 269 students (125 treatment and 144 comparison) who qualified 
for intervention in fourth grade (mean age = 9.57 years; SD = 0.70 years) based on a reading 
comprehension score at or below the 30" percentile on the Gates-MacGinitie Reading Test 


(GMRT; MacGinitie, MacGinitie, Maria, Dreyer, & Hughes, 2006) and also participated in the 
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follow-up testing in the fall of fifth grade. Students in the follow-up sample came from 11 public 
elementary schools located in four school districts at sites in the southeastern and southwestern 
United States. Fifty-one percent of students were male, 20% were identified as having limited 
English proficiency, 50% were identified as low income, and 12% were identified as having a 
disability. Fifty two percent of students identified their ethnicity as Hispanic. The sample was 
also racially diverse with 42% White, 32%, Black, 21% American Indian, 1% Asian, and 4% 
multiracial. 

Data Collection Procedures and Measures 

The students were tested on the reading comprehension measure in the fall of fourth 
grade prior to intervention, in the spring of their fourth-grade year after intervention completion, 
and once more in the fall of their fifth-grade year after a 10 week summer break. All testing was 
completed by test administrators trained by the research team. Test administrators were blind to 
study conditions. 

The reading comprehension subtest of the Gates MacGinitie Reading Tests (MacGinitie 
et al., 2006) is a group-administered, norm-referenced test. Students independently read brief 
passages and answer a series of multiple-choice questions about each reading. The questions 
cover facts, inferences, and drawing conclusions. Students have 35 min to complete the test. 
The test-retest reliability for fourth grade students is above .85. Internal consistency is .96. 
Construct validity estimates for the Gates range from .79-.81 with the earlier addition of the test, 
the Preliminary Scholastic Assessment (PSAT), the Scholastic Assessment Tests (SAT), and the 
American College Testing Program (ACT). 


Intervention Procedures 
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The intervention is fully described in the efficacy study (Wanzek et al., 2017). We 
provide a summary of the procedures and components here for reference to the current study. 
Students with reading comprehension difficulties in the fall of first fourth grade were randomly 
assigned to receive the Passport intervention (treatment) or to receive typical school services 
(comparison). The students assigned to the treatment condition received the Passport 
intervention for 30 min daily in groups of 4-7 students throughout the fourth grade school year, 
typically during the school’s designated intervention time. Students received a mean of 92 
sessions (mean of 2,760 min) over the course of the school year. 

Passport is a multicomponent reading intervention that includes instruction in phonics 
and word recognition, vocabulary, comprehension, and fluency. Lessons are grouped in 10-day 
units with overarching themes (e.g. Faraway Places). Each lesson of Passport is divided into two 
major components: Word Works and Read to Understand. Instruction during Word Works 
focuses on phonics and word recognition and word reading fluency. During Word Works, 
teachers lead students in word study on letter sound correspondence and common letter patterns, 
affixes, and common syllable types. This portion of the lesson includes instruction and practice 
in reading and spelling decodable words, irregular words, and using syllabication and affixes to 
read multi-syllabic words. The second portion of the lesson, Read to Understand, focuses on 
instruction in vocabulary, comprehension, and includes some fluency practice with connected 
text. This portion of the lesson includes a selected text that alternates between fiction and 
nonfiction throughout the lessons. Students are taught how to preview text and make 
predictions. Instruction also includes modeling and practice with comprehension strategies such 
as main idea, making inferences, drawing conclusions, summarizing, and comprehension 


monitoring. During the first six weeks of instruction, each lesson consists of 20 min of Word 
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Works followed by 10 min of Read to Understand. After the first six weeks of instruction, the 
focus of the lesson shifts and Word Works is reduced to 5 min with 25 min devoted to Read to 
Understand. 

Training and Fidelity of Implementation. Ten trained intervention tutors who were 
members of the research team administered the intervention. All tutors had bachelors’ degrees, 2 
tutors also had master’s degree, and 1 tutor was in the process of obtaining a doctorate degree. 
Five of the tutors were certified teachers. All tutors were female. Three tutors identified their 
ethnicity as Hispanic. Six tutors were White, 3 were Black, and 1 declined to identify her race. 

All tutors were trained in the implementation of the intervention by project coordinators 
before intervention sessions began. Training was completed in 8 hr, over 2 days and included 
instruction in the goals of the program, instructional routine, and practice implementing various 
components. In addition to the initial training, project coordinators completed two coaching 
visits each month with each tutor and held monthly meetings to review implementation of 
intervention components and provide continued support. 

Implementation fidelity data for the treatment were collected once per month for each 
intervention tutor. Observers rated each lesson component for fidelity of implementation, quality 
of implementation, and student engagement. Implementation was rated on a 0 to 3-point scale 
with 0 indicating the component was not implemented and 3 indicating all or nearly all of the 
required elements were implemented. Engagement and instructional quality were also rated 
from 1 (weak engagement or quality) to 3 (excellent engagement or quality). Instructional 
quality indicators included ongoing monitoring, redirection of off-task behavior, positive and 
corrective feedback, organization of materials, and appropriate selection of additional items for 


practice when needed. All observers established 90% or higher levels of agreement with the first 
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author before completing observations. Implementation of the intervention was consistently 
high. Mean implementation ratings of the intervention components ranged from 2.82 to 3.00 for 
each intervention tutor. Mean quality ratings ranged from 2.76 to 2.97 for each tutor. Mean 
student engagement ranged from 2.82 to 2.95 for each intervention tutor. Implementation further 
verified that instruction focused on developing students’ reading comprehension and vocabulary 
(62% of intervention time). 

Typical School Services. All students participated in Tier I, core classroom reading 
instruction as required. This instruction was largely focused on reading comprehension and 
vocabulary development (46% of time) and largely provided in a whole class grouping (60% of 
time). As part of school-delivered RTI services, a total of 103 students (51 in treatment and 52 
in comparison group) received supplemental reading instruction provided by their school in 
addition to their core reading instruction at some point during the school year. This instruction 
also focused on reading comprehension and vocabulary (45% of the time). Instructors reported 
intervention sessions to be 5 to 30 min in length for 14% of students, 31 to 50 min for 57% of 
students and more than 50 min for 29% of students. The supplemental intervention was 
delivered by the classroom teacher 8% of the time, other certified teachers 44% of the time, 
paraprofessionals or volunteers 37% of the time, speech-language pathologists or speech 
therapists 9% of the time, or via video technology 2% of the time. 

Data Analysis 

Where the previous study reported significant effects of Passport treatment stemming 
from latent constructs of reading comprehension (Wanzek et al., 2017), the study was focused on 
simple pretest-posttest collection efforts. As the current study was focused on the extent to 


which the effects of the intervention were maintained into the next grade level following 
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cessation of intervention implementation, we used a series of multilevel growth models to 
identify the rate of change and whether change was discontinuous between the treatment and 
comparison groups. Whereas many studies of follow-up effects simply focus on between-group 
analyses testing the efficacy of the intervention at the individual follow-up occasion, we opted to 
use piecewise growth models and treat the effect of the intervention as a time-varying covariate 
(McCoach & Kaniskan, 2010). When modeling data from a two-group, two-wave efficacy 
design, a categorical variable is typically created that represents the grouping of students as 
participating in the intervention group (i.e., coded as “1”) or the comparison group (i.e., coded as 
“O)”. This coding scheme is invariant over time meaning that the comparison group is always 
coded as 0, 0 for the pretest and posttest and the intervention group is always coded as 1, 1 for 
the pretest and posttest. By coding the grouping variable as a time-varying covariate (TVC) ina 
growth model, it is possible to test how the treatment effect changes from pretest to posttest and 
posttest to follow-up. It is important to note that in a growth model that treats group as a TVC, 
there are, minimally, two covariates in the growth model: 1) the effect of time (e.g., 0, 1, and 2 
for three wave design), and 2) the TVC effect of group. 

One example of a plausible coding design of the grouping variable as a TVC is when 
testing for persistent treatment effects (PTE). The PTE model supposes that the effect from 
pretest to posttest is fully maintained from posttest to follow-up; participants in the comparison 
group would be coded as 0, 0, 0 for all three time points because at no time was there treatment 
for those students. Individuals in the treatment condition would be coded as 0 at the pretest as no 
intervention was provided then, 1 at the posttest to denote the change from pretest to posttest, 
and 1 at follow-up to denote that the effect observed from pretest to posttest was fully 


maintained. An alternative coding design for the grouping variable as a TVC is for 
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circumstances where researchers hypothesize a diminishing treatment effect (DTE) from posttest 
to the follow-up assessment. As in the PTE model, the comparison group is coded as 0, 0, 0 for 
all three waves and the intervention group is coded as 0, 1, 0 to reflect that the observed effect 
from pretest to posttest fully diminished to baseline at the third time-point. A special case of the 
DTE is that one may hypothesize that the treatment effect diminishes but not fully to baseline. 
For example, the observed data patterns might suggest that the effect diminishes by 50% in 
which case the intervention group would be coded as 0, 1, .5. Another observed data pattern 
might suggest that the treatment effect diminishes by 75% resulting in a coding of 0, 1, .25 for 
the intervention group. These examples serve to highlight that treating the coding of the 
grouping variable for the intervention group is flexible to allow for an empirical test of how the 
effect should best be modeled. 

Five separate, multilevel growth models of students nested within classrooms and schools 
were tested to determine the functional form of growth and the extent to which treatment effects 
persisted. Model 1 included only time as covariate (coded as 0, 1, 2 for pretest, posttest, and 
follow-up) but treated the random effects of intercept and slope as fixed across all nesting units. 
Model 2 included random effects for the intercepts across all levels and fixed slopes; Model 3 
included random intercepts and slopes across levels. The best fitting growth model among these 
three comparisons was selected by comparing the AIC and sample adjusted BIC indexes (nBIC), 
Models 4 and 5 then separately added a TVC testing for a persistent treatment effect (i.e., Model 
4) or a diminishing treatment effect (Model 5). The AIC and nBIC were used for choosing the 
final growth model between Models 4 and 5. Although the nature of the data was such that 


students were longitudinally cross-classified, classroom and school information was not 
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collected at the follow-up period, thus the original nesting during the intervention study was used 
to account for non-independence of observations. 
Results 

Descriptive Statistics 

A review of data completeness revealed that no missing data were observed at either the 
pretest or the follow-up and 2% of the data were missing from the posttest. Little’s test of data 
missing completely at random (MCAR) was not statistically significant, y?(2) = 0.09, p > .500, 
indicating that the data met MCAR criteria. Full information maximum likelihood was used to 
account for the missing data in the growth models. Descriptive statistics for the sample are 
reported in Table 1 and demonstrated that the groups were approximately equivalent in their 
pretest score (Passport M = 437.37, SD = 21.05; Comparison M = 440.06, SD = 18.15; Hedges’ 
g = -0.14). Consistent with previously published findings, the treatment group significantly 
outperformed the comparison group at posttest, F(2,263) = 16.02; Hedges’ g = 0.16, with the 
treatment group showing an approximate 20 point gain from pretest to posttest compared to an 
approximate 15 point gain for the comparison group. Moreover, at the fifth grade follow-up, the 
treatment group maintained a descriptive advantage in GMRT reading comprehension 
performance (M = 462.22, SD = 23.18) compared to the comparison group (M = 460.02, SD = 
25.57) with each group growing approximately five additional scaled points over the summer, 
though this difference was not statistically significant. 
Intervention Follow-up Effects 

Table 2 provides the fit for the three baseline growth models: 1) the fixed intercept and 
fixed slope (Model 1); 2) random intercept and fixed growth (Model 2); and 3) random intercept 


and random slope (Model 3). Comparisons among these three baseline growth showed that 


Retention of Reading Intervention 17 


Model 2 fit better than Model 1 based on both the AIC (AAIC = 88.80) and the nBIC (AnBIC = 
101.50). For context, a AnBIC of at least 5 is considered to be a practically important difference 
between models (Raftery, 1995). Moreover, Model 3 fit better than Model 2 on both indices 
(AAIC = 13.20; AnBIC =17.20); thus, the best fitting model was one that included random 
intercepts and slopes (i.e., Model 3; Table 2). The specific configuration of the random intercept 
and slopes model included random intercepts at the student and school levels along with random 
slopes at the classroom and school levels. Using this basic growth model, we then examined the 
persistent and fleeting effects models. Model 4 included the TVC for persistent treatment effects 
and Model 5 included the TVC for a diminishing effect. In order to best optimize the coding for 
the TVC in Model 5, the proportional mean score differences over time based on observed scores 
in the treatment group were evaluated. Table 1 shows that from posttest to follow-up, the 
treatment group changed an average of 5.01 points. Relative to their mean change from pretest 
to posttest (i.e., 19.84), the mean posttest to follow-up change was 25% of the original observed 
change (i.e., 5.01/19.84). As such, we coded the TVC in Model 5 as 0, 1, .25 to reflect a 75% 
diminished growth estimate. Both Models 4 and 5 fit better than Model 3 by values of at least 5, 
and comparisons between Models 4 and 5 in Table 2 pointed to the diminished treatment effects 
(DTE) model providing better fit to the data compared to the persistent treatment effects (PTE) 
model (AAIC = 4.10, AnBIC = 4.10) Figure 1 plots the observed scores as well as the predicted 
group estimates from the DTE model (Table 3) showing that where groups were equivalent at 
baseline on the predicted scores and the treatment group outperformed the comparison group at 
the posttest on the predicted scores, the treatment group did not maintain growth at their initial 
change level, preventing the treatment group from further outperforming the comparison group 


at the follow-up assessment. The estimated treatment effect at the follow-up assessment was g = 
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0.14, and compared to the post-test treatment effect of g = 0.16, reflects the fact even where the 
growth trajectory from post-test to follow-up was fleeting, or deviating from an expected growth 
trajectory, the standardized difference between the groups was largely maintained from the post- 
test (0.16) to follow-up (0.14). 
Discussion 

The purpose of this study was to examine and quantify the retention of reading 
comprehension effects of a fourth grade reading intervention across the summer break into the 
fifth grade year. We found that, on average, the reading comprehension effects of the 
intervention were partially fleeting. On average, students in the treatment continued to grow at a 
rate of approximately 25% of the original change they made during the fourth grade intervention. 
The students in the comparison group made similar gains to the treatment group over the 
summer. Notably, if the intervention had a persistent effect, we would have observed students in 
the treatment continuing to grow at a faster rate over the summer relative to the comparison 
group. We had hypothesized a partially fleeting effect of the intervention due to the lack of 
instruction or intervention between the grades when the intervention resource faucet was turned 
off. As with many schools in the United States, there was no summer school available to the 
students in this study sample. Thus, based on previous research related to summer learning loss, 
the fleeting effect is to be expected, and the finding that the effect was partially fleeting (25% of 
the original growth trajectory continued over the summer) is positive. Similarly, the fact that the 
comparison students also did not experience summer learning loss, but continued to grow is also 
positive. 

Relative to the comparison group, students still gained an advantage in reading 


comprehension from the Passport intervention overall. Although there was a partially fleeting 
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effect of the intervention over the summer (25% of the original growth trajectory), our study 
adds to the research base on RTI/MTSS by demonstrating that the students in the treatment 
group began fifth grade with the same advantage over the comparison group in reading 
comprehension that they had demonstrated at posttest. In other words, they maintained the 
accelerated reading comprehension growth from fourth grade (grew at the same rate as the 
comparison group over the summer), but did not continue to show accelerated growth over the 
comparison group in reading comprehension over the summer. The small group, supplemental, 
multi-component reading intervention increased student reading comprehension over and above 
typical school services and this effect was retained into the fifth grade year when the intervention 
resources faucet was tured back on for the students, allowing the treatment group to begin fifth 
grade at a higher reading comprehension level than they otherwise would have without the 
intervention. This is positive news for the treatment group’s ability to close the gap with grade 
level expectations over time. These students could continue in the RTI/MTSS process without 
having lost ground they had gained from a previous intervention. It is also notable that students 
in this sample (treatment and comparison) did not experience summer learning loss in reading 
comprehension, particularly given the large percentage of students with low socioeconomic 
status (Alexander et al., 2001; Benson & Borman, 2010) and the fact that no summer school 
programming was provided. 

Fleeting effects of reading intervention, particularly over summer breaks, are an 
important conversation as we consider RTI/MTSS models. A student may be a responder to an 
intervention at the end of one grade level and arrive back at school for the next grade level 
demonstrating lower learning levels, requiring intervention simply to reach previous 


achievements. The lack of summer programming for the current sample of students no doubt 
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contributed to the partially fleeting effects of the intervention. In fact, when no summer 
programming is available, as was the case with the current sample, it is reasonable to expect a 
fully fleeting effect (no further growth during that time period). The fact that these students with 
reading difficulties experienced only a partially fleeting effect, 25% of their original growth, in 
their accelerated learning despite the summer break is a positive finding. Of course, the most 
efficient learning for students with learning difficulties would occur with an intervention that not 
only accelerates student learning, but also demonstrates high levels of persistent effects when the 
intervention is completed. The question is whether these interventions exist, particularly 
between grades when the intervention resources are turned off for several months and students 
experience a break without continued instruction. A variety of home and school summer 
programs intended to keep that instructional faucet running have shown positive effects in 
reading (Kim & Quinn, 2013). As a field we may need to consider the potential role of summer 
interventions within RTI/MTSS models in order to serve students with reading difficulties more 
effectively and efficiently. 

Overall, the Passport intervention allowed students with reading difficulties to accelerate 
their reading comprehension and retain that learning into the next grade level. As a result, this 
group of students can begin fifth grade instruction and intervention with levels of reading 
comprehension that are similar to where they ended fourth grade, allowing for efficient learning 
in fifth grade interventions to further close the gap with grade level expectations. The students in 
the comparison group were also able to maintain their reading comprehension levels from fourth 
grade, albeit at a lower level than students in the treatment group. 


Limitations 
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The students in this sample did have a few weeks of school prior to their fifth grade 
testing which may have improved some of their learning from the summer (Hill et al., 2007). 
Testing in the first week of school may have shown effects that were even more fleeting. 
However, the treatment and comparison students were tested during the same weeks of school so 
they had equal opportunity to any instruction. In addition, schools were still doing their own 
testing and no supplemental interventions had yet begun. Future research could assist in 
determining whether students gain back any fleeting effects of intervention from the summer and 
how long it takes to gain these effects back after they resume instruction. 

We examined retention and fleeting effects for one specific intervention. The findings 
are specific to the Passport intervention, but do provide the field with a comparison as the 
summer effects of other reading interventions are quantified. We also did not collect information 
about students’ independent summer reading, number of books, magazines, or e-books or 
computer/technology-based reading programs available in the home, or whether students 
accessed public libraries. We argue that educators need information on effective and efficient 
reading interventions that will allow students to accelerate their learning during the intervention 
and retain that learning, if not continue to grow, particularly across grades that often includes a 


significant break without instruction. 
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Table 1 


Descriptive statistics for Passport and Comparison group by assessment wave 


GMRT Reading Comp. 

Condition Time point N Mean SD Mean Difference 
Passport Pretest 125 437.37 21.05 

Posttest 124 457.21 24.99 19.84 

Follow-Up 125 462.22 23.18 5.01 
Comparison Pretest 144 440.06 18.15 

Posttest 140 454.73 23.75 14.67 

Follow-up 144 460.02 25.57 5.29 


Note. GMRT = Gates MacGinitie Reading Test. 
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Table 2 


Fit index comparison for growth models 
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Model AIC nBIC Deviance 
1. Fixed Intercept & Growth 7305.4 7310.1 7303.4 
2. Random Intercept/Fixed Growth 7216.6 7208.6 7208.6 
3. Random Intercept & Growth 7203.4 7191.4 7191.4 
4. Persistent Effect (PTE) WAGL2. «7792 FAT9.2 
5. Diminishing Effect - 75% (DTE) 7187.1 7175.1 FATS A 


Note. AIC = Akaike Information Criteria, nBIC = sample-adjusted Bayes Information Criteria 
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Table 3 
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Fixed and random effects from Fleeting — 75% treatment effects model 


Fixed Effects Estimate SE df t-value p-value 
Intercept 439.72 1.37 799 321.4 <.001 
Time 10.88 0.94 799 11.56 <.001 
Flee75 7.59 2.08 799 3.62 <.001 
Random Effect Estimate SE z-value p-value 

Student Intercept 81.49 33.17 2.46 0.007 

Student Slope 0 

Teacher Intercept 0 

Teacher Slope 5.54 899.42 0.59 0.278 

School Intercept 8.62 5.15 1.67 0.095 

School Slope 2.94 4.56 0.65 0.259 

Residual 337.08 20.97 16.08 <.001 


Note. The G-matrix for the students slope and teacher intercept random effects was a non- 


positive definite resulting in a near-0/0 estimate. 


SE = Standard Error. Df = degrees of freedom. Flee75 = Fleeting at 75% . 
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Figure 1. Comparison of observed and predicted means by each time-point for Passport and 


Comparison groups. 


29 


